Here we provide a descriptive overview of the syllable identifications relative to target (table 1). In the current data the number of syllables identified by EasyAlign perfectly matched the targeted number of syllables, i.e., in 100% of the trials there were 0 differences in the number of syllable detected versus target. Note however that we will manually check the EasyAllign syllable detections for the final dataset.
Table 1. Percentage of syllable detection mismatches| syllable differences | percentage |
|---|---|
| 0 | 100.00 |
Table 2 provides the percentages of different type of L2 stress placement matches and mismatches.
Table 2. Percentage of correct L2 placements| stress mis/match type | stress difference | percentage |
|---|---|---|
| L2 correct | same | 33.93 |
| L2 incorrect & L1 match | same | 0.00 |
| L2 incorrect & L1 mismatch | same | 16.07 |
| L2 correct | difference | 44.05 |
| L2 incorrect & L1 match | difference | 0.00 |
| L2 incorrect & L1 mismatch | difference | 5.95 |
For the first analysis we simply assess whether the absolute difference in stress timing relative to the target stress time is different for the gesture or the no gesture condition.
D$accuracy <- abs(D$stressed_mistimingL2L1) #absolute deviation from stress from L2
#basemodel predicting the overall mean accuracy
model0 <- lme(accuracy~1, data = D, random = list(~1|ppn, ~1|target), method = "ML", na.action = na.exclude)
#alternative model with gesture versus no gesture as predictor
model1 <- lme(accuracy~condition, data = D, random = list(~1|ppn, ~1|target), method = "ML", na.action = na.exclude)
anova(model0, model1) #test difference basemodel versus model 1## Model df AIC BIC logLik Test L.Ratio p-value
## model0 1 4 4069.241 4084.509 -2030.620
## model1 2 5 4066.132 4085.217 -2028.066 1 vs 2 5.109432 0.0238sum1 <- summary(model1) Click here for model 1 summary
```r
sum1
```
```
## Linear mixed-effects model fit by maximum likelihood
## Data: D
## AIC BIC logLik
## 4066.132 4085.217 -2028.066
##
## Random effects:
## Formula: ~1 | ppn
## (Intercept)
## StdDev: 0.01107632
##
## Formula: ~1 | target %in% ppn
## (Intercept) Residual
## StdDev: 58.9668 85.65074
##
## Fixed effects: accuracy ~ condition
## Value Std.Error DF t-value p-value
## (Intercept) 62.19643 8.046698 167 7.729435 0.0000
## conditiongesture -21.28571 9.373199 167 -2.270912 0.0244
## Correlation:
## (Intr)
## conditiongesture -0.582
##
## Standardized Within-Group Residuals:
## Min Q1 Med Q3 Max
## -1.7144678 -0.4332536 -0.1847360 -0.1847359 4.1253449
##
## Number of Observations: 336
## Number of Groups:
## ppn target %in% ppn
## 2 168
```It is of course possible that there is a contextual effect of gesture, depending on whether there is a stress difference or a presence of an accent (see Figure 2)
Figure 2.
We will further assess this in a complex model we expand our analysis with the relevant stimuli conditions, as well as their interactions with the gesture condition. If the interactions are statistically reliable we will perform a post-hoc comparisons with R-package “lsmeans” with a bonferroni correction.
#alternative model with gesture versus no gesture as predictor
model2 <- lme(accuracy~condition+stress+accent, data = D, random = list(~1|ppn, ~1|target), method = "ML", na.action = na.exclude)
model3 <- lme(accuracy~condition+stress+accent+
condition*stress+
condition*accent,
data = D, random = list(~1|ppn, ~1|target), method = "ML", na.action = na.exclude)
anova(model1, model2, model3) #test difference basemodel versus model 1## Model df AIC BIC logLik Test L.Ratio p-value
## model1 1 5 4066.132 4085.217 -2028.066
## model2 2 7 4067.404 4094.124 -2026.702 1 vs 2 2.727168 0.2557
## model3 3 9 4070.265 4104.619 -2026.132 2 vs 3 1.139483 0.5657#summary model 3 post hoc
sum3 <- summary(model3)
posthoc3a <- lsmeans(model3, list(pairwise ~ condition|stress), adjust="bonferroni")
posthoc3b <- lsmeans(model3, list(pairwise ~ condition|accent), adjust="bonferroni")Click here for model 3 summary
```r
sum3
```
```
## Linear mixed-effects model fit by maximum likelihood
## Data: D
## AIC BIC logLik
## 4070.265 4104.619 -2026.132
##
## Random effects:
## Formula: ~1 | ppn
## (Intercept)
## StdDev: 0.009244317
##
## Formula: ~1 | target %in% ppn
## (Intercept) Residual
## StdDev: 58.19645 85.36076
##
## Fixed effects: accuracy ~ condition + stress + accent + condition * stress + condition * accent
## Value Std.Error DF t-value p-value
## (Intercept) 62.29167 13.93053 165 4.471592 0.0000
## conditiongesture -9.94048 16.27766 165 -0.610682 0.5423
## stressdifference -18.29762 16.08559 164 -1.137516 0.2570
## accentaccent present 18.10714 16.08559 164 1.125675 0.2619
## conditiongesture:stressdifference -3.00000 18.79582 165 -0.159610 0.8734
## conditiongesture:accentaccent present -19.69048 18.79582 165 -1.047599 0.2964
## Correlation:
## (Intr) cndtng strssd accntp cndtn:
## conditiongesture -0.584
## stressdifference -0.577 0.337
## accentaccent present -0.577 0.337 0.000
## conditiongesture:stressdifference 0.337 -0.577 -0.584 0.000
## conditiongesture:accentaccent present 0.337 -0.577 0.000 -0.584 0.000
##
## Standardized Within-Group Residuals:
## Min Q1 Med Q3 Max
## -1.60812794 -0.46910650 -0.23332169 -0.08683539 4.28520679
##
## Number of Observations: 336
## Number of Groups:
## ppn target %in% ppn
## 2 168
```Click here for posth-hoc3a output
```r
posthoc3a
```
```
## $`lsmeans of condition | stress`
## stress = same:
## condition lsmean SE df lower.CL upper.CL
## nogesture 71.3 11.4 1 -73.2 216
## gesture 51.6 11.4 1 -93.0 196
##
## stress = difference:
## condition lsmean SE df lower.CL upper.CL
## nogesture 53.0 11.4 1 -91.5 198
## gesture 30.3 11.4 1 -114.3 175
##
## Results are averaged over the levels of: accent
## Degrees-of-freedom method: containment
## Confidence level used: 0.95
##
## $`pairwise differences of condition | stress`
## stress = same:
## 2 estimate SE df t.ratio p.value
## nogesture - gesture 19.8 13.3 165 1.489 0.1385
##
## stress = difference:
## 2 estimate SE df t.ratio p.value
## nogesture - gesture 22.8 13.3 165 1.714 0.0883
##
## Results are averaged over the levels of: accent
## Degrees-of-freedom method: containment
```Click here for posth-hoc3b output
```r
posthoc3a
```
```
## $`lsmeans of condition | stress`
## stress = same:
## condition lsmean SE df lower.CL upper.CL
## nogesture 71.3 11.4 1 -73.2 216
## gesture 51.6 11.4 1 -93.0 196
##
## stress = difference:
## condition lsmean SE df lower.CL upper.CL
## nogesture 53.0 11.4 1 -91.5 198
## gesture 30.3 11.4 1 -114.3 175
##
## Results are averaged over the levels of: accent
## Degrees-of-freedom method: containment
## Confidence level used: 0.95
##
## $`pairwise differences of condition | stress`
## stress = same:
## 2 estimate SE df t.ratio p.value
## nogesture - gesture 19.8 13.3 165 1.489 0.1385
##
## stress = difference:
## 2 estimate SE df t.ratio p.value
## nogesture - gesture 22.8 13.3 165 1.714 0.0883
##
## Results are averaged over the levels of: accent
## Degrees-of-freedom method: containment
```We perform a mixed linear regression with normalized acoustic markers as DV, and acoustic marker (peak F0, peak envelope, and duration) x condition as independent variable.
Figure x.
Dlong <- gather(D, "marker", "acoust_out", 13:15)
#alternative model with gesture versus no gesture as predictor
model0 <- lme(acoust_out~1, data = Dlong, random = list(~1|ppn, ~1|target), method = "ML", na.action = na.exclude)
model1 <- lme(acoust_out~marker*condition, data = Dlong, random = list(~1|ppn, ~1|target), method = "ML", na.action = na.exclude)
anova(model0, model1) #test difference basemodel versus model 1## Model df AIC BIC logLik Test L.Ratio p-value
## model0 1 4 1951.533 1971.196 -971.7666
## model1 2 9 1534.941 1579.182 -758.4703 1 vs 2 426.5926 <.0001#summary model 3 post hoc
sum1 <- summary(model1)
posthocsum1 <- lsmeans(model1, list(pairwise ~ condition|marker), adjust="bonferroni")Click here for model 1 summary
```r
sum1
```
```
## Linear mixed-effects model fit by maximum likelihood
## Data: Dlong
## AIC BIC logLik
## 1534.941 1579.182 -758.4703
##
## Random effects:
## Formula: ~1 | ppn
## (Intercept)
## StdDev: 1.375216e-05
##
## Formula: ~1 | target %in% ppn
## (Intercept) Residual
## StdDev: 0.08539542 0.506822
##
## Fixed effects: acoust_out ~ marker * condition
## Value Std.Error DF t-value p-value
## (Intercept) 1.5512255 0.03977187 835 39.00308 0.0000
## markerpeakF0z -0.5074599 0.05546413 835 -9.14933 0.0000
## markersDURz -0.7038124 0.05546413 835 -12.68951 0.0000
## conditiongesture 0.1850414 0.05546413 835 3.33624 0.0009
## markerpeakF0z:conditiongesture -0.1713849 0.07843812 835 -2.18497 0.0292
## markersDURz:conditiongesture -0.3458323 0.07843812 835 -4.40898 0.0000
## Correlation:
## (Intr) mrkrF0 mrkDUR cndtng mrkF0:
## markerpeakF0z -0.697
## markersDURz -0.697 0.500
## conditiongesture -0.697 0.500 0.500
## markerpeakF0z:conditiongesture 0.493 -0.707 -0.354 -0.707
## markersDURz:conditiongesture 0.493 -0.354 -0.707 -0.707 0.500
##
## Standardized Within-Group Residuals:
## Min Q1 Med Q3 Max
## -4.32238038 -0.52000609 0.00753372 0.54513919 4.64779877
##
## Number of Observations: 1008
## Number of Groups:
## ppn target %in% ppn
## 2 168
```Click here for posthoc model 1
```r
posthoc3a
```
```
## $`lsmeans of condition | stress`
## stress = same:
## condition lsmean SE df lower.CL upper.CL
## nogesture 71.3 11.4 1 -73.2 216
## gesture 51.6 11.4 1 -93.0 196
##
## stress = difference:
## condition lsmean SE df lower.CL upper.CL
## nogesture 53.0 11.4 1 -91.5 198
## gesture 30.3 11.4 1 -114.3 175
##
## Results are averaged over the levels of: accent
## Degrees-of-freedom method: containment
## Confidence level used: 0.95
##
## $`pairwise differences of condition | stress`
## stress = same:
## 2 estimate SE df t.ratio p.value
## nogesture - gesture 19.8 13.3 165 1.489 0.1385
##
## stress = difference:
## 2 estimate SE df t.ratio p.value
## nogesture - gesture 22.8 13.3 165 1.714 0.0883
##
## Results are averaged over the levels of: accent
## Degrees-of-freedom method: containment
```In the previous analyses we know whether speech prosody performance increases or decreases as a function of gesture, stress difference, and accentedness. A further question is whether gesture-speech synchrony is affected by stress difference and accentedness.
Figure 3.
It seems that synchrony between gesture and speech is best in the accented and stress match condition.
```
## Model df AIC BIC logLik Test L.Ratio p-value
## model0 1 4 2044.734 2057.230 -1018.367
## model1 2 6 2047.489 2066.233 -1017.745 1 vs 2 1.2451018 0.5366
## model2 3 7 2049.444 2071.312 -1017.722 2 vs 3 0.0448689 0.8322
```Now we should know whether gesture-speech synchrony can be affected by trial condition that may complicate correct stress placement. If indeed gesture-speech synchrony is affected, we can wonder about how gesture and speech diverge when they are more asynchronous. Firstly assess whether gestures
#basemodel predicting the overall mean accuracy
model0 <- lme(asynchrony_L2L1~1, data = subD, random = list(~1|ppn, ~1|target), method = "ML", na.action = na.exclude)
#alternative model with gesture versus no gesture as predictor
model1 <- lme(asynchrony_L2L1~stress*correct, data = subD, random = list(~1|ppn, ~1|target), method = "ML", na.action = na.exclude)
anova(model0, model1) #test difference basemodel versus model 1## Model df AIC BIC logLik Test L.Ratio p-value
## model0 1 4 2133.622 2146.118 -1062.811
## model1 2 7 2126.648 2148.515 -1056.324 1 vs 2 12.97427 0.0047summary(model1) ## Linear mixed-effects model fit by maximum likelihood
## Data: subD
## AIC BIC logLik
## 2126.648 2148.515 -1056.324
##
## Random effects:
## Formula: ~1 | ppn
## (Intercept)
## StdDev: 0.007961735
##
## Formula: ~1 | target %in% ppn
## (Intercept) Residual
## StdDev: 130.1349 2.115984
##
## Fixed effects: asynchrony_L2L1 ~ stress * correct
## Value Std.Error DF
## (Intercept) 14.14516 16.72969 163
## stressdifference 30.74958 22.54347 163
## correctL2 incorrect & L1 mismatch 49.44575 32.69010 163
## stressdifference:correctL2 incorrect & L1 mismatch -210.84048 58.87324 163
## t-value p-value
## (Intercept) 0.845512 0.3991
## stressdifference 1.364013 0.1744
## correctL2 incorrect & L1 mismatch 1.512560 0.1323
## stressdifference:correctL2 incorrect & L1 mismatch -3.581262 0.0005
## Correlation:
## (Intr) strssd cLi&Lm
## stressdifference -0.742
## correctL2 incorrect & L1 mismatch -0.512 0.380
## stressdifference:correctL2 incorrect & L1 mismatch 0.284 -0.383 -0.555
##
## Standardized Within-Group Residuals:
## Min Q1 Med Q3 Max
## -0.063643564 -0.007597103 -0.002391494 0.008405780 0.050266435
##
## Number of Observations: 168
## Number of Groups:
## ppn target %in% ppn
## 2 168https://link.springer.com/article/10.3758/s13428-021-01546-0